Ham Radio 2000 #1/Ham Radio 2000.iso/ham2000/packet/terminal/paket61/man09.doc

Text File | 1994-08-15 | 17.8 KB | 449 lines

PART IX - paKet PROTOCOL (pP)

What is pP ?

  paKet Protocol, or "pP" for short, is a YAPP-compatible Binary File
  Transfer protocol designed especially to provide a Recovery capability
  after an aborted file transfer. It is basically the YAPP Protocol with
  some additional enhancements to provide the automatic restart capability.

  Consider the situation where you are sending a Binary File to another
  station and the transfer is aborted for some reason, whether it be a bad
  link or power failure or any other reason. You simply send the file
  again, using the same <F7> command!  With pP, the file transfer will
  automatically start from where the previous transfer left off.

  pP will mostly be appreciated when you are transferring a large file and
  the transfer is aborted when almost finished. For small files, or if a
  transfer is aborted near the beginning, you might as well send the
  entire file again.

  pP may be used to exchange Binary Files with any other station equipped
  with either pP or YAPP protocol. The pP protocol automatically detects
  the presence of another pP station and will use the enhanced features as
  required, otherwise the standard YAPP protocol will be used.

  I used the YAPP Protocol Specifications published by Jeff Jacobsen
  (WA7MBL), the author of the original YAPP program, when developing this
  pP system. Extensive testing has shown pP to be fully compatible with
  YAPP Protocol.

  At this stage, pP is available only with paKet.  However, the paKet
  Protocol details are offered to the Public Domain and are discussed in
  some detail below. I will be happy to assist developers of other YAPP
  Protocol software with the pP enhancements so they too can offer this
  feature in their software.

  But please note, I am offering to discuss the pP *ENHANCEMENTS* to
  *EXISTING* YAPP Protocol programs. Please don't call me for assistance
  with the development/debugging of Standard YAPP Protocol software!

  In the sections that follow, I give some examples of the exchange of
  dialog between the two stations. For convenience here, I will refer to
  the two sides as the Sender and the Receiver. So, if I use the term
  "Sender" or "Receiver" I might be referring to the operator or to the
  software or to the system generally, but I am sure you will get the idea.

  I hope Jeff Jacobsen will forgive me if I suggest his original YAPP
  program has now been superseded but his YAPP Protocol lives on as the
  current Standard for Binary File transfers in the world of Packet Radio.
  In the following sections, when I refer to "YAPP", I am referring to the
  YAPP Binary File Transfer Protocol, not to the original YAPP program
  that started it all.


                               Page 309

Using pP

  For most Binary File transfers, pP will behave in a similar manner to
  YAPP and the performance will be the same.

  No special action is required to use pP.  Using pP is the same as using
  YAPP - the procedure to send or receive a Binary File is the same.

  Refer to the <F7> and <F8> keys in the "Keyboard Commands" section of
  this Manual for operational details.


 Typical scenario:

  Say you are sending a Binary File to another pP station. You send the
  file in the usual way, starting with the <F7> key.

  The transfer begins just like any other Binary File transfer starting
  from the beginning of the file.

  Then part way through, let's say after sending half the file, something
  goes wrong.  Ummm, for this illustration we'll say the digipeater we
  were using goes off the air and we lose the link with the other station.

  So, call up using a different path and establish communications with
  that station again.  Then start the same file transfer again, using the
  same <F7> key again, specifying the same filename again. Come to think
  of it, this is what you would do anyway, isn't it? Whether using pP or
  not! But with most other protocols, the entire file would be resent from
  the beginning. With pP, a Recovery is possible.

  When the file transfer is started again, the following activity takes
  place (assuming both stations support pP):

     1. The exchange of pP/YAPP initialisation codes verifies each station
        is ready to start the transfer (standard YAPP Protocol).

     2. The Sender sends the Protocol header, which includes the file name
        and file size (standard YAPP protocol). The header also includes a
        new field to identify this station as a pP station.

     3. The Receiver detects that this file already exists on that system
        and is smaller than the one about to be received. (The file is
        smaller because it is only half there!)

        The Receiver now knows that the Sender is a pP station because of
        the additional field in the header.

     4. The Receiver requests a pP recovery and sends some details of its
        (shorter) file.

     5. The Sender checks those details against the file about to be sent.
        It looks like the same file, so the Sender issues an approval then
        starts sending from the agreed point in the file.
                               Page 310


        (If the Sender decides it does not look like the same file, the
        Recovery request would be denied and the transfer would start from
        the beginning).

     6. When the Receiver gets the Recovery approval, that system will
        reset the file pointers and will start writing any data received
        at the agreed starting point in the file, effectively appending to
        the existing file.

     7. The transfer then continues from that point using standard YAPP
        Protocol.

  While all this is going on, you do nothing - just watch the messages
  informing you of the exchange of pP dialogue as outlined above. Then
  when the Binary File Transfer Window appears you will see the file
  transfer is already partly completed! It has picked up almost from where
  it left off.




































                               Page 311

The technical details of the paKet Protocol:

  For this discussion I will be referring to some of Jeff Jacobsen's YAPP
  Specifications. If you are a developer of YAPP compatible software, you
  will already know about these things. If you are an interested user and
  do not have these specifications handy, I will try to explain as I go.

  A warning though... this section does get a bit heavy and is included
  for the technically minded. If you find you are having trouble
  understanding it, just skip to another section - it is NOT necessary to
  understand all this in order to use pP.


Sender   - Sending a pP/YAPP Header

  The YAPP Header is listed in the YAPP specifications as follows:
  HD    Send_Hdr
  <SOH> <len> (Filename) <NUL> (File Size in ASCII) <NUL> (Opt)

  (The "< >" combination is used to indicate single byte values).

  This header is sent at the beginning of the transfer to identify the
  name and size of the file about to be sent. Note there is an optional
  field provided, and it is this optional field that pP uses to identify
  the Sender as a pP station.

  When the Sender sends a file header, the "optional" field will contain
  the pP Identifier string, "paKet-Protocol" (without the quotes).

  So if we were sending a file "PROG.EXE" and that file was 74692 bytes
  long, the pP header might look like:
     <SOH><29>PROG.EXE<NUL>74692<NUL>paKet-Protocol

  This optional field should be sent with every Header. If the Receiver is
  running Standard YAPP Protocol it will not get upset because the
  optional field is provided for in the Specifications. Testing with
  various YAPP Protocol systems has shown this works without problems,
  however should some system somewhere object to this optional field being
  used in this way, the paKet user can switch off the pP option in the
  Configuration - File Transfer options. In that case the Sender would not
  send the optional field and the Receiver will think we are running
  Standard YAPP Protocol!  I have yet to find a system that misbehaves so
  you would normally leave the pP option ON in your Configuration.


Receiver - Receiving a pP/YAPP Header and requesting Recovery

  When the Header is received, the Receiver will check the Header's file
  details. If that file does not yet exist it will create a new file, send
  an <ACK> to the Sender and continue with the transfer using Standard
  YAPP Protocol.

  If a file of the same name already exists on the Receiver system it
  could be just another version of the same file or maybe it is the result
                               Page 312

  of an earlier transfer attempt that was not completed. If it is a partly
  completed copy of an earlier transfer attempt, pP can identify this and
  perform a Recovery. If it is not that same file, pP can recognise this
  too and revert to Standard YAPP Protocol for a "normal" transfer of the
  entire file.

  The Receiver will request a pP Recovery if the following conditions are
  ALL met:
      1. a file already exists with the same file name;
                              and
      2. The size of the existing file is greater than 1000 bytes;
                              and
      3. The file size is smaller than that in the Header just received;
                              and
      4. the Header contains the "paKet-Protocol" identifier;
                              and
      5. the user of this Receiver system has indicated in the
         Configuration that pP may be used.
         (This last condition is implemented in paKet but is an optional
         feature of the pP Protocol).


  Let's discuss briefly each of these requirements.

  1. If the file does not yet exist, there is no need for Recovery and a
     standard YAPP Protocol transfer will take place.

  2. You will see later that a Recovery does not actually begin from the
     end of this short file, but begins 750 bytes back from the end. If
     the Receiver's file is shorter than 1000 bytes, it is hardly worth a
     Recovery anyway!  We might just as well start again from the
     beginning.

  3. If the Receiver's file is already bigger than the one about to be
     sent, it is certainly NOT the result of an aborted transfer of this
     same file!

  4. If the Header does not contain the pP identifier, the Sender is not
     running pP. There is no point in asking a non-pP system for a
     Recovery!

  5. The paKet user may disable the pP facility, so the ultimate control
     remains with the user. This option is provided in paKet but is not a
     requirement of the Protocol. Another program that supports pP might
     not provide this user option.

  If all the above conditions are met and the Receiver decides to request
  a Recovery, a "NP" record is prepared and sent to the Sender. This NP
  record is defined below:





                               Page 313

 NP record.

  The YAPP Specifications include an NR record for a negative
  acknowledgement ("Not Ready" sometimes referred to as NAK). The standard
  NR record provides for the <NAK> code followed by an optional "reason"
  in ASCII.

  Using this format with the optional field to maintain compatibility
  with the Standard YAPP Protocol, I have created a new Protocol record
  and I called it NP. This new record is:


  NP    pP Recovery request
  <NAK><len>paKet-Protocol,(length of file),(first 100 bytes)(last 100 bytes)

  - The <NAK> is the ASCII code with a value of 21;

  - <len> is the length of the following data;

  - The pP identifier string "paKet-Protocol" comes next (14 bytes).
    This is the optional "reason" provided for in the YAPP Specifications!;

  - The length of the file is actually the length of the file on the
    Receiver's system LESS 850 bytes. (Further discussion on this below)

    The value is sent as ASCII characters and please note there are commas
    before and after these numbers.

    So, if the file on the Receiver's system is 53,964 bytes, this field
    would contain the following seven ASCII characters:
       ,53114,

  - The first 100 bytes of the short file on the Receiver system;

  - The "last" 100 bytes of the short file on the Receiver system.

  To get the "last" 100 bytes the Receiver system will move to a position
  850 bytes back from the end of the file and read the next 100 bytes
  from there. So the last 750 bytes will be ignored and if a Recovery is
  successful, those 750 bytes will be overwritten.

  This figure of 850 bytes is somewhat arbitrary. I felt it was necessary
  to provide SOME overlap to cover the situation where the end of the
  short file might be corrupted (it WAS aborted for some reason, don't
  forget!). If you are developing your own pP system, feel free to choose
  another figure - this affects only the Receiver, so your system can
  still perform a Recovery from paKet (or any other pP system). I would
  suggest a larger rather than a shorter figure should be chosen if you
  wish to change it. You might like to reduce the overlap to save time on
  duplicating the data already sent, but keep in mind it takes only a
  second or two to send 100 bytes and the Recovery could save many
  minutes, maybe an hour! I don't think the user will get upset if it
  takes a few seconds to Recover.

                               Page 314

  So, as an example, an NP record might look like:
     <NAK><221>paKet-Protocol,53114,........
  (the dots represent the 200 data bytes).

Sender   - Processing a Recovery request (NP record) from the Receiver

  After sending the Header, the Sender would usually get an RF packet
  indicating the other station is now ready to "Receive the File". In that
  case the entire file is sent from the beginning in accordance with the
  Standard YAPP Protocol.

  If the Sender gets the NP packet, it means the other station is
  requesting a pP Recovery. It could be they already have part of this
  file so it is now up to the Sender to decide whether it is or is not the
  same file. Of course the only way to be really sure about this is to
  compare the entire file but if we are going to do this we might as well
  send the whole thing again!

  I decided it is a reasonable risk to compare the first 100 bytes of the
  two files, then to compare the last 100 bytes of the files. (The "last"
  100 bytes in this case would be somewhere in the middle of the Sender's
  file.) If these 200 bytes match - same values at the same locations and
  the two files have the same name, then pP will accept they ARE the same
  file and the Recovery will be approved.

  The NP packet includes the size of the file on the Receiver's system,
  the first 100 bytes plus the last 100 bytes of that file. So the Sender
  system will access the file about to be sent and will compare the first
  100 bytes. If they are all the same, it will move down the file to the
  location corresponding to the file size given in the NP record. If the
  next 100 bytes in the file also match those given in the NP record, a
  Recovery will be approved.

  If even one byte does not match, the Recovery will be denied.

  So, if the Sender approves the Recovery, it will send the "Approval for
  Recovery" packet, the "AP" record, which is a new Protocol record and
  consists of two bytes:
     <ACK><6>
  Then it will position the file pointer to the file size given by the NP
  record plus 100 bytes. (Those "last" 100 bytes have been matched so
  there's no need to send them again.)  And the file transfer continues
  from that point using Standard YAPP Protocol.

  If the Sender denies the Recovery, it will send the "Recovery Denied"
  packet, the "DN" record. This too, is a new Protocol record and consists
  of 16 bytes to distinguish it from an "ordinary" NAK:
     <NAK><14>paKet-Protocol
  As the Recovery has been denied, the file will be sent in its entirety
  and the transfer will proceed using Standard YAPP Protocol, starting
  from the beginning of the file.



                               Page 315

Receiver - Processing Response to Recovery request.

  This is the final stage of the Recovery process. The Receiver had
  requested a Recovery and now will find out whether the Recovery is
  approved or denied. The Receiver will get either an Approval (AP record)
  or a Denial (DN record). Details of these records are shown in the
  previous section.

  If the Recovery is approved, the Receiver will adjust the file pointer
  to the file size less 750 bytes.

  Yes, that's right 750! We moved back 850 bytes into the file to include
  the "last" 100 bytes in the the NP record. Now that we have an approval,
  we know those "last" 100 bytes are OK so there's no need to transfer
  them again. We will start receiving data from AFTER those "last" 100
  bytes, overwriting the last 750 bytes in the file.  (See comments above
  under "NP Record" - you may vary this "setback" figure of 850 bytes if
  you are developing your own pP software).

  If Recovery is denied, the entire file will be sent so the Receiver will
  either have to overwrite the existing file or create a new one.

  I would like to point out here that if the paKet program is receiving a
  file and Recovery is denied, the existing (short) file will be preserved
  and a new file with a different extension (eg: .000 or .001 etc) will be
  created. This is a feature of the paKet program NOT part of the pP file
  transfer Protocol.  pP just manages the Recovery process. So if some
  other program uses pP it might handle "Recovery Denied" differently by
  simply producing an error message such as "File Exists" or it might
  overwrite the existing short file.
























                               Page 316